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ABSTRACT 



This report is based on the collective research of seven 
regional education laboratories related to the federally funded Comprehensive 
School Reform Demonstration program. The participating laboratories were the 
Appalachia Education Laboratory (AEL) in partnership with the Center for 
Research in Educational Policy (CREP) at the University of Memphis, Lab at 
Brown University Education Alliance, Northwest Regional Laboratory, 
Mid-Continent Research for Education and Learning, Pacific Resources for 
Education and Learning, SouthEastern Regional vision for Education, and 
WestEd. Each conducted its own research project designed for different 
purposes, used different methodologies, and reflected the different 
perspectives of the researchers and contexts of each laboratory region. The 
report presents commonly identified themes that relate to the processes used 
by schools to plan for comprehensive school reform and how those processes 
supported program implementation. A key area in several of the projects was 
the amount of teacher participation in selection of reform models and its 
effects on implementation. The findings were mixed. Two laboratory projects 
explicitly examined the effect of model selection on reform implementation 
and school climate using similar instruments, but used different definitions 
of the constructs . Researchers at AEL/CREP found statistically significant 
differences in school climate and implementation progress between schools 
using different model-selection processes. Contradictory findings may be due 
to differences in conceptualizations and methodologies, but also to 
differences in school or district leadership and/or regional cultures. An 
appendix contains project summaries. (Contains 12 references.) (DFR) 
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EXECUTIVE SUMMARY 



This report is based on the collective research of seven Regional Education Laboratories 
related to the federally-funded Comprehensive School Reform Demonstration program. The 
participating laboratories are the Appalachia Education Laboratory (AEL) in partnership with the 
Center for Research in Educational Policy (CREP) at the University of Memphis, Lab at Brown 
University Education Alliance (LAB), Northwest Regional Education Laboratory (NWREL), 
Mid-continent Research for Education and Learning (McREL), Pacific Resources for Education 
and Learning (PREL), SouthEastem Regional Vision for Education (SERVE), and WestEd. Each 
participating laboratory conducted its own research project which were designed for different 
purposes, used different methodologies, and reflected the different perspectives of the 
researchers and contexts of each laboratory region. In an effort to examine key issues across the 
different research projects, the participants in this research initiative agreed to apply data 
gathered in their research to address four general areas of CSRD implementation. This report 
presents the common themes identified that relate to the processes used by schools to plan for 
comprehensive school reform and how those processes supported program implementation. The 
common themes identified and reported are organized by level of government: federal- and state- 
level, district-level, and school-level. 

Interaction of Federal and State Policies with Model Selection 

As encouraged in the Comprehensive School Reform Demonstration (CSRD) program 
legislation, resources from Title I and CSRD were combined to support comprehensive school 
reform in some schools. Two states considered here used CSRD to support other state initiated 
reform efforts. These state efforts limited schools’ choice of models but also facilitated 
coordination between CSRD and other reform activities. State imposed time limits for submitting 
CSRD grant applications reduced the ability of some schools to gather information about 
designs, to disseminate that information within schools, and to involve their school communities 
in planning for the application. As part of the CSRD planning process, many applicants appeared 
to rely on needs analyses that were mandated by separate state or district policies. 

District Support of Planning 

Districts supported the development of CSRD applications with data analysis, grant 
writers, data collection from model design teams, links with the community, and assistance to 
schools in integrating school and district standards or goals for the proposal. Districts also 
provided administrative support, such as negotiating with model service providers and help with 
budgeting. As an interested party and a key source of information about reform designs, districts 
could also reduce the choice of reform models available to schools. Limiting the model choices 
available to schools was sometimes a purposeful outcome of district-level needs assessments. 

School Planning Activities 

A key area in several of the laboratory’s research projects was the amount of teacher 
participation in reform model selection and its effects on implementation. The findings were 
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mixed. Two laboratory projects explicitly examined the effect of model selection on reform 
implementation and school climate using similar instruments, but used different definitions of 
the constructs. Researchers at AEL/CREP found statistically significant differences in school 
climate and implementation progress between schools using different model selection processes. 
While researchers at SERVE observed some similar patterns in their data, different statistical 
analysis techniques and variant definitions of the constructs were used. Using qualitative 
research methods, researchers at two other laboratories gathered data on the selection processes. 
McREL reported that teachers appeared to have (gradually) embraced principal-mandated 
reforms, while researchers at LAB at Brown reported faculty frustration caused by a principal’s 
top-down decisions. These contradictory findings may be due to differences in 
conceptualizations and methodologies, but also to differences in school or district leadership 
and/or regional cultures. 

Finally, the work at NWREL focused on schools that were not prepared for reform and 
described factors that influence a school’s readiness to implement CSRD including leadership, 
vision, high expectations, and accountability. 

Next Steps 

The relationship between district roles, the model selection process, and implementation 
is an area of continuing research across the regional laboratory system. Given what is being 
learned about school preparedness for reform, a key direction for this continuing work is likely to 
be an examination of appropriate district roles given different school-level capacities to decide 
on and implement reforms. 
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INTRODUCTION 



In fiscal year 2000, Congress appropriated $220 million for the Comprehensive School 
Reform Demonstration (CSRD) program (PL 105-78), a $70 million increase over appropriations 
in each of the previous two fiscal years. The majority of CSRD money is distributed to schools 
through state-run grant competitions to support local efforts to improve entire school operations 
and student achievement. The program also supports related research and technical assistance 
conducted at the Regional Education Laboratories (RELs). 

The intent of CSRD has been to expand the quality and quantity of schoolwide reform 
efforts, to leverage ongoing efforts to connect higher standards with school improvement 
(especially for Title I schools) and to encourage coordination of all resources and school 
functions to meet identified needs. Many previous targeted and categorical federal programs for 
raising student achievement have tended to increase fragmentation and generally have failed to 
raise student achievement (Millsap, Millsap, Turnbull, Moss, Brigham, Gamse, & Marks, 1992; 
Puma, Jones, Rock & Fernandez, 1993; U.S. Department of Education, 1993). The CSRD 
program has been consciously designed to avoid such problems by using proven comprehensive 
approaches. 

For students, the promise of CSRD is an improved education created when their school 
leaders engage in coherent, coordinated, schoolwide planning to implement strategies and 
programs with proven effectiveness. The work presented here — the work of the seven RELs 
participating in this research initiative — has been to broadly inquire into how this promise is (or 
is not) being realized in a variety of schools, districts and states across the U.S. It is based upon 
information gathered during the laboratories work as CSRD technical assistance providers and 
researchers. 

THE CROSS-LABORATORY CSRD RESEARCH PROJECT 

The recently published national evaluation of CSRD stressed the need for continuing 
research into comprehensive reform (U.S. Department of Education, 2000). As part of that effort, 
each of the laboratories has been working with states, districts and schools as they implement 
CSRD. 



The work presented here is based on the research activities of several of these 
laboratories. Each of the participating laboratories focused on different levels of the educational 
system, including state-, district-, and school-levels. 

Reflecting the different training of the researchers involved and the different regional 
contexts’ involved, each laboratory has adopted different purposes, perspectives, methodologies, 
and contexts for their work. These different approaches and methodologies 1 are summarized in 



' Other methods and studies may have been part of CSRD research at the participating laboratories, but they were 
not part of this project. 
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Table 1. For example, both LAB at Brown and PREL report findings relevant to CSRD 
implementation at the state level. The LAB at Brown’s research method has been mainly 
ethnographic and relied on participant observation, interviews, reflection, and document review 
to detect patterns. PREL’s work has focused on providing technical assistance, and has relied on 
results of self-study instruments to provide data upon which patterns or implementation were 
detected. Similarly, while several laboratories report findings relevant to CSRD implementation 
at the school level, each research approach and focus was different. PREL’s approach was to 
determine needs for technical assistance across 10 sites and to respond to these needs. NWREL’s 
approach was to provide intensive, individualized assistance to two low-performing schools to 
prepare them for comprehensive school reform. The approach at AEL/CREP and SERVE was to 
survey teachers and administrators systematically using standardized classroom observation and 
student assessment tools, and compare outcomes across different school climates, models and 
programs. McREL’s approach to the study of CSRD implementation was to interview 
administrators about supports for CSR implementation and inspect responses for patterns. A 
more detailed description of each laboratories research project is contained in the Appendix to 
this report. This document and other products of the laboratories collaboration reflect the benefits 
of combining multiple research methodologies but also the logistical challenge of combining 
studies that have different, if overlapping, structures and premises. 

Table 1. 

Summary of CSRD Research Included in This Report 



REL 


Research Focus Methodology Observations 


AEL/CREP 


CSRD Implementation Process Survey & Case Studies 12 schools 


LAB at 

Brown 


State role in implementing CSRD Ethnography & Applied 2 states, 20 schools 

Research 


NWREL 


Preparation for and Case studies 2 schools 

implementation of reform in high 
needs schools 


McREL 


CSRD implementation, literacy Case studies 9 schools 

and district role 


PREL 


Identify school needs and provide Questionnaires and 10 schools 

technical assistance Interviews 


SERVE 


CSRD Implementation Process Survey & Case Studies 39 schools 


WestED 


Role of Districts in CSRD Computer aided analysis of 9 districts 

interviews 



The participating researchers came together in April 1999 to learn about different reform 
perspectives and contexts and to inform the process of school improvement in ways beyond any 
individual effort. Since each researcher had already designed and begun conducting studies to 
address different questions, a considerable amount of time was spent deciding on a process that 
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would allow a coherent discussion of what was learned from the independent experiences and 
research. 

A number of different approaches to working together were considered: (1) writing a 
book or monograph to which each would contribute a chapter, (2) developing or adopting a 
common rubric for measuring implementation success across CSRD sites, and (3) identifying a 
common set of areas to which each participant would respond based on their experiences and 
data. The group settled on the third option and generated the following set of four a set of 
common questions: 

1. The planning process used to develop each school’s CSR plan and how 
did the process support CSR implementation, 

2. The relationship between leadership and CSR implementation, 

3. The conditions that supported/inhibited professional learning, 

4. The story you want to tell about CSR models, and its implications for 
policy. 

The first question was selected as the focus of this cross-laboratory project. Each 
laboratory prepared a 10-15-page response to the question that focused specifically on issues of 
culture/climate, strategies and activities, decision making, external technical assistance, state 
guidelines and policy, and federal guidelines and policy. This document reports the common 
themes identified from these responses. In many cases, whole paragraphs from an individual 
laboratory’s responses have been reproduced here, with the appropriate attribution. 

Following a brief overview of the CSRD planning literature, the results section of the report 
is organized in view of a CSRD planning scheme shown in Figure 1. Essentially, complementary 
research is reported for multiple levels of education systems. That is, the results section begins 
by describing links between CSRD and Title 1 Programs, then discusses state-level planning, 
followed by discussion of district-level activities, and finally activities at the school level are 
reported. 



SCHOOL WIDE REFORM PLANNING LITERATURE 

The literature on planning for school-wide reform identifies many of the themes that are 
also present in the laboratory research presented here, particularly with respect to information 
dissemination and participatory decision making at the school level. 

The state has a considerable influence on setting the stage for schools to undertake 
comprehensive school reform. Many states have limited the models that schools can choose to 
implement (ECS, 1999). States play a role in providing schools with adequate time and 
autonomy to investigate the various models to ensure that the reform model selected is an 
effective and appropriate match for a school’s needs and capabilities (ECS, 1999; Bodilly, 1998). 
Hurried start-up and lack of preparedness can lead to a mismatch between a school and the 
chosen design which then can contribute to poor implementation in the long run (Glennan, 
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1998). Lack of a reform at the state level can ultimately work against school-level reforms (Lusi, 
1997). To assist schools most in need with planning and model selection, many states have 
provided programs for low-performing schools that lack the capacity and leadership to begin this 
process. Furthermore, some state education agencies have taken it upon themselves to assist 
schools in choosing a model by evaluating prospective models to make sure that they are 
compatible with state and district standards, analyzing research, encouraging schools to make 
site visits to other schools, and conducting workshops to help determine if a model is a match for 
the school (ECS, 1999). 




Figure 1 : Schema of CSRD Planning (Dashed lines show relationships that are not addressed 
in this document) 

Earlier research, especially work with the New American Schools, has found that district 
support was essential for schools when they begin a comprehensive school reform planning 
process. First, observed leadership from the state and district level can be a strong catalyst for 
change as well as reassuring to teachers (Bodilly, 1998). An especially strong message of support 
at the district and school level is the reallocation of resources to support the planning process 
(Bodilly, 1998). In addition, most schools require guidance and leadership from the state and 
district offices regarding the skills and knowledge that they need and are expected to teach 
students (Glennan, 1998; McREL, 1999). One approach has been to provide professional 
development to school staff on how to conduct a needs assessment and how to work as a team 
(ECS, 1999; SERVE, 1997). With regard to selecting a model, districts have allocated time and 
built time into teachers’ schedules so that they can participate in the planning process. This has 
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proven conducive to more successful implementation (Haynes, 1998). Some districts have also 
held design fairs to present schools with a variety of models and information, have had model 
representatives come to schools and give presentations, and have funded site visits to other 
schools using a given model that is being considered (Slavin, 1997). 

The literature suggests that school-level planning should begin with a needs assessment 
that includes an evaluation of student test score and demographic data (ECS, 1999; McREL, 
1999). The success or failure of a reform program relies heavily on teachers, the change agents 
who will actually implement the program (McREL, 1999). Therefore, it is not surprising that 
research suggests that teachers need to be involved in the decision-making process from the 
beginning. This includes involving teachers in goal setting, studying different models, and 
interviewing model developers (ECS, 1999). This “orientation” phase is vital to establish staff 
support for mission, goals, and philosophy of the programs to be implemented (Haynes, 1998). 
School personnel can also visit demonstration sites in order to make an informed decision about 
the appropriateness of a specific model (Glennan, 1998; Slavin, 1997). A communication 
breakdown during this stage, whether it is a poor understanding of the design, staff resistance, or 
not enough information, can contribute to a mismatch between the program and the school 
(Glennan, 1998). Furthermore New American Schools research has shown schools that adopted a 
model without fully understanding it, or schools that did not have a voice in what model they 
chose, showed lower levels of implementation than those that were well informed and had 
substantial involvement in the decision-making process (Bodilly, 1998). 

CROSS-LABORATORY FINDINGS RELATED TO CSRD PLANNING 

Federal Level Finding 

Direct Link Between CSRD and Title 1. Title I schoolwide programs have served as a 
source of encouragement to develop and adopt comprehensive reform programs. These programs 
have also offered resources that were used to network and gather information about reform 
designs, ultimately supporting CSRD applications. As was the intent of CSRD legislation, it 
appeared that in some schools, principals were combining resources from Title I and CSRD to 
support comprehensive reform. 

The State and Federal Consolidated Grants Official in one of the mid-size districts 
reported that five years ago, the district embraced Title I schoolwide programs, 
moving all schools into the program. He stated “the district valued comprehensive 
school reform and started doing it on its own anyway. Supporting school 
applications for CSRD allowed the district to continue supporting and guiding 
schools in a process they already had underway. (McREL) 

In two sites, the Title I program played a networking role as evident by reports 
that the Title I coordinator learned about the CSRD program at a conference and 
recommended to principals that they apply for the CSRD program grants. He then 
assisted with the process. Other mention of Title I influence on CSR planning was 
that Title I funds were used to support professional development related to the 
CSR program. (McREL) 
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CSRD and Title I schoolwide initiatives in Puerto Rico and New Hampshire appeared to 
be complementary. Title I was seen as initiating comprehensive reform with CSRD resources 
being used to support and strengthen ongoing reform efforts. 

The relative contribution to comprehensive reform from CSRD and Title I are not 
clear. Among the Lightspan schools (in Puerto Rico) that LAB at Brown 
researchers visited the school change processes are comprehensive, but are that 
way because of use of comprehensive Title I schoolwide plans. CSRD funds have 
allowed the schools to ‘shore up’ a few areas of identified weakness. (LAB at 
Brown) 

LAB at Brown researchers, who facilitated a workshop on local evaluation 
strategies for CSRD schools in New Hampshire, heard leaders from three of the 
six schools acknowledge the role of Title I schoolwide planning in initiating their 
comprehensive school reform processes. CSRD was seen as providing more 
targeted resources after the whole-school change process was well underway. 

(LAB at Brown) 

State-Level Findings 

Integrating CSRD With Ongoing State Initiatives. The national evaluation noted that the 
implementation of CSRD has been “accompanied by a great deal of state activity” (U.S. 
Department of Education, 2000). In the research done at the participating laboratories, at least 
two states used CSRD funds to support other reform efforts. Elsewhere, CSRD implementation 
within a state was shaped by educational priorities identified at the state level. This can be seen 
as a type of policy alignment. 

(Maine decided) ... to concentrate its limited CSRD resources all at the high 
school level, to integrate CSRD with Promising Futures (1998) — its statewide 
secondary level reform initiative — and to include non-CSRD participating 
schools in CSRD-related professional development. (LAB at Brown) 

California used the CSRD program as part of an overall effort to aid low-performing 
schools. 

California’s approach to comprehensive reform is a multi-pronged effort 
supported by the Public Accountability Act of 1999. This Act includes the federal 
government’s CSRD program as well as statewide efforts to ensure the maximum 
of under-performing schools receive the necessary resources to formulate and 
implement school wide plans for reform. (WestEd) 

In California if a district was unable to present a compelling plan to support its 
schools, all schools in that district would be ineligible for CSRD grants. 
(WestEd). 

In a third jurisdiction, Puerto Rico, educational priorities declared by the Commissioner 
of Education clearly also influenced how that island chose to rollout CSRD. 
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The Commissioner called for particular emphases on improving instruction in 
English and incorporating technology into the classroom. While neither Lightspan 
nor the Computer Curriculum Consultants models that were adopted in more than 
two-thirds (58 of 75) of the selected CSRD schools in Puerto Rico are 
comprehensive on their own, both respond directly to the Commissioner’s 
priorities. (LAB at Brown) 

State Role in CSR Model Selection: Guiding School Choices. The model parameters in 
the federal CSRD request for proposals allowed states to provide guidance on the models schools 
could select. A clear example of this was Maine where a waiver was obtained from the U.S. 
Department of Education to limit CSRD implementation to the secondary level and to add 
parameters for applicant schools to assure consistency with the state’s high school reform 
framework Promising Futures. In effect, Maine required interested model providers to illustrate 
how their models matched the core practices embraced in their statewide reform framework. 

The most formative decision that shaped Maine’s CSRD implementation design 
was the decision by Maine’s Commissioner of Education to use CSRD as a 
vehicle to promote Promising Futures. Because of this decision, only secondary 
schools in Maine were eligible for CSRD funding. It also meant that Maine had 
created a framework ensuring that all of its CSRD schools and their plans were 
sufficiently similar such that the recipient schools could aid and learn from each 
others’ implementation efforts. Whether this was initially an intended outcome of 
the CSRD adaptation decision is unclear, but it created a feature of Maine’s 
CSRD program (similarity of design) that the state program coordinator has 
positively exploited .... In a very real sense, Maine created its own CSRD 
model — Promising Futures — that was consistent with at least one of the 
catalogued CSRD models — the Coalition of Essential Schools. (LAB at Brown) 

Time and Its School-Level Impacts. A very clear method of increasing or decreasing the 
model options open to schools was through the amount of information available to schools 
concerning models. A simple, but powerful mechanism that can limit the available information 
to schools is the amount of time schools have to complete their CSRD proposal (U.S. 
Department of Education, 2000). This time can be limited by short deadlines set by states issuing 
the RFP, or slow mechanisms within districts for alerting schools of the funding opportunity. It 
should be noted that the original federal timeline for CSRD rollout in all states was very 
ambitious. While some states were able to meet this timeline, federal education policy-makers 
relaxed the original requirements. 

Limited planning time reduced the ability of schools to gather information about designs, 
disseminate that information within schools, and to involve a full range of their school 
communities in CSRD proposal preparation. As evident below, schools in some of these states 
also reported that they felt rushed by state guidelines. Because of such hurried preparation, 
several unsuccessful applicant schools studied expressed relief at the failure of their bids. 

In the first round of Maine funding all schools faced what the state CSRD 
coordinator now characterizes as a too short timeline. ...the school had had about 
a five-week window between hearing about CSRD at a state-sponsored forum and 
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when the complete application was due. While this principal felt that her 
stewardship of the proposal was one of the finest leadership moments of her 
career, other faculty in her building emphasized and she conceded that the 
proposal had generated resistance that still lingered. Faculty doubt was articulated 
as “who are they [the feds/the state] to tell us what to do,” “CSRD is too top- 
down,” and so on. Indeed, almost a year after hearing that they would not be 
funded, there was still palpable bitterness related to CSRD as a top-down program 
and a sense that the school had “dodged a bullet” by having its application denied. 

(LAB at Brown) 

.... the hurry-up schedule for grant development meant that only a small team of 
faculty wrote the grant... Because the original proposal drafting timeline at this 
school had not lent itself to broad faculty involvement, many of the uninvolved 
faculty felt little compulsion to heed the grant proposal or the Promising Futures 
framework. In a sense, after their grant proposal was not funded, the principal 
sided with this uninvolved majority, resisting both the state framework, as well as 
those on his own faculty who had figured out a way to customize and use the state 
framework. (LAB at Brown) 

In contrast, some states, by design or political coincidence, were slower to rollout CSRD. 
The extra time for planning was used advantageously by some schools and districts. 

For the second round of funding in Maine, the request for proposal was made 
available eight months before the applications were due, and the state CSRD 
coordinator organized several workshops to support the development of 
applications. (LAB at Brown) 

The request for proposals was released in April of 1999. Districts and schools 
received technical assistance support on writing the grants from April through 
July. RFPs were due to the state department on July 30. Grants were awarded in 
October of 1999. Because the federal program was initiated in other states prior to 
the spring of 1999 both the schools and districts in California were aware of the 
program and it’s potential to support reform efforts well in advance of the release 
date. In fact, almost all districts in our sample reported “waiting” for California to 
announce its competition. (WestEd) 

In anticipation of the release of the request for proposals many districts began 
working with schools to help staff learn about the nine components of the 
program and to provide information on different research-based models of reform. 

Once the competition was announced districts stepped up their support to assist 
schools prepare submissions for the July deadline. (WestEd) 

District-Level Findings 

Needs Analysis: Supported By Districts And States. It appears that many grant 
applicants included a needs analysis as part of their planning process. In the McREL region, 
ongoing needs assessments, done to meet state mandated accreditation requirements, helped 
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schools identify needs and select designs. A number of laboratory studies examined the types of 
needs assessments used by schools. In general, however, it was not possible to determine if the 
needs analysis was used in CSRD planning or added to the grant applications post facto. 

In all nine sites (in the MCREL study), an ongoing school improvement planning 
process or a needs assessment was used to develop the comprehensive school 
reform plan. The needs assessments were both formal and informal. In some mid- 
size districts, accountability systems alerted central office personnel to needs. In 
another district, central office personnel became generally dissatisfied with the 
reading performance of children across the district in the primary grades. This 
dissatisfaction led to a district wide reading initiative which involved piloting a 
CSR model in four of the district’s schools. (McREL) 

Ongoing school improvement processes were mentioned explicitly by respondents 
in three sites when explaining how they became involved in the CSRD program. 
Respondents at one rural Kansas school reported that an application to the CSRD 
program was a natural outgrowth of the state Quality Performance Accreditation 
(QPA), the school-by-school improvement/accreditation process. In another site, 
the school improvement plan developed to address the schools low performance 
became the basis of its CSRD application. In a third site, a respondent attributed 
their CSR implementation progress to their school improvement planning process. 
(McREL) 

Based on data from the principal interview, teacher focus group, and application 
documents, all 12 CSRD schools included in the (AEL/CREP) research study 
used some type of needs analysis process as reported in Table 2. (AEL/CREP) 



Table 2. 

Needs Analysis Components Used At Schools 

N=12, 1999-2000 (AEL/CREP) 



Needs Analysis Components 


Number of Schools 
Reported Using The Component 


Student achievement data 


12 


Deficiencies in current curriculum 


7 


Teacher need for specific training 


7 


Amount and degree of parental involvement 


5 


School/community demographics 


12 


School Improvement Plan 


7 


District plans/goals 


3 



However, based on these data, it was difficult to determine if the needs analysis 
information was actually used during the planning and model selection process or 
if it was simply added to the document post facto. (AEL/CREP) 
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District Support of School Applications. There was a wide variety in the amounts and 
methods of district support provided to schools. This support ranged from assisting with 
information gathering to helping schools complete tasks associated with the CSRD applications. 

All districts reported a supportive environment from district staff personnel in 
facilitating the development of CSR applications. Some schools reported they 
were most interested in receiving hands-on support from district staff, whereas 
others wanted more autonomy to reflect and develop their plan. (WestEd) 

In order to make CSRD a priority, district staff showed support to schools in 
many different ways. Some districts provided funding for school staff to visit 
model developer fairs to learn about the different kinds of researched-based 
models and the ways in which they would work to support their individual 
school’s goals for reform. Others hosted fairs at their district for schools. Many 
districts hired grant writers to coordinate the process of both preparing and 
writing the grant. The majority of schools in our sample spoke of the ample 
support their districts provided facilitate their application process. (WestEd) 

The amount of support provided to schools during the planning and model 
selection process varied greatly by school. In some schools, the support provided 
by the district was substantial. (AEL/CREP) 

States and districts expanded the number of models from which schools could choose by 
providing information on multiple models through mechanisms such as model fairs. Districts 
have further supported expanding the number of model choices by providing resources for visits 
by school personnel to demonstrations, presentations and schools implementing the design. 

From our initial analyses we learned that schools found district support was most 
helpful when districts 

• had an existing infrastructure for reform, 

• hosted design fairs to introduce different models to schools, 

• provided funding for schools to visit presentations and demonstrations of 
model implementation, and 

• assigned district personnel to attend professional development trainings 
with school staff. (WestEd) 

One strategy used in the planning and model selection process was to provide 
educators with additional exposure to CSRD model(s). Techniques used to 
provide this exposure varied by school. The most popular technique was for a 
team from the school to visit other school sites using a specific CSRD model. The 
team observed classrooms and asked questions of the faculty and staff. Seven 
schools (of 12) used this approach.... Another approach designed to provide 
educators with more exposure to various models was a “model fair.” The fair 
consisted of multiple model developers presenting their models to teachers from 
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numerous schools. Videos and information packets were available for teachers, as 
well as the opportunity to ask specific questions about the model. Teams of 
teachers from two schools (of 12) used this approach.... Other approaches 
included attending a model seminar or watching a video. (AEL/CREP) 

For the majority of sites (7 of 9), districts appeared to provide access to and 
dissemination of information about the CSRD program and model providers. In 
the mid-size districts, demonstration sites served as proof that CSR models were 
effective or were identified as pilot sites to examine feasibility and effectiveness 
of models. (McREL) 

Some districts also financially supported school personnel visits to demonstration 
sites outside the district. Frequently, principals acknowledged the helpful 
assistance of district personnel with preparing and writing the CSRD application 
and providing supporting data. (McREL) 

.... the Hawaii Department of Education (HCDOE) stated an intention to provide 
technical assistance in the form of CSRD information dissemination. (PREL) 

Although districts (and states) worked to provide schools with opportunities to gather 
information on designs, it is not clear that information from these events was actually used in 
model selection. 

Showcases of CSR model providers were hosted collaboratively by State 
Education Agencies (SEAs) and the Regional Educational Laboratory. 
Occasionally, however, knowledge about CSR model providers and their 
characteristics appeared to be shaped more by word of mouth than by research 
findings. (McREL) 

Similar to the state departments of education, districts could also reduce the available 
model choices to schools. Limiting school choices was sometimes an outcome of needs 
assessments conducted by the district. Perceptions of school-level needs led some district 
administrators to direct schools towards specific types of reform such as literacy, to supporting 
specific models such as Success for All and The Learning Network , or to develop local models. 
Districts also directed schools to specific designs for fiscal reasons or for the perceived good of 
the district at large. 

Four schools either did not mention support from the district, or the support was 
more limited.... In the three remaining schools (of 12), external assistance was 
provided by the district, but included more than just support. In one school, 
district representatives practically mandated which model the school would 
adopt... In the other two schools, the county developed a CSR model for use in 
the schools. (AEL/CREP) 

District central office personnel attended conferences or showcases seeking 
information on CSR models and then made recommendations and disseminated 
the information to school personnel. (McREL) 
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In more directive districts, central office personnel appeared to sway CSR model 
selection with financial reasons or for the good of the district at large, perhaps 
deliberately studying different options to determine how best to meet the needs of 
students. One of our nine sites was piloting The Learning Network, as were three 
other non-CSRD schools in the district, in response to dissatisfaction with primary 
grade children's reading. In another district, when asked what were the identified 
needs for the school improvement efforts at the school working with Lightspan, a 
central office administrator explained that English Language Learners needed 
extended learning time and Lightspan was considered an alternative, less 
expensive look-a-like to Extended Day for providing curriculum reinforcement. 
(McREL) 

In some instances districts were very hands-on with helping schools select their 
reform models. In three cases, only one model was presented to schools as a 
possible option whereas in the other districts five or more models were provided 
as options. In general, district staff tended to perceive their input as simply advice 
and not direction. However, the schools perceived it quite differently. In our 
school interviews, staff stated that the districts narrowed their options. This 
suggests the districts should be more aware of the power they exert through 
proposing possible model choices for schools. (WestEd) 

Districts supported CSRD applications with analysis, through hiring grant writers, 
gathering information from model design teams, liaison with the community, and helping 
schools integrate school and district standards/goals for the proposal. 

From initial analyses of our research we learned the technical assistance for 
schools both in preparation for and implementation of CSR plans was best aided 
by districts when district staff 

• worked to clarify the grant, its nine components, and the process of 
preparing such a large document, 

• hired or used in-house staff to support schools in the writing of CSR plans, 
and 

• assigned staff to work with schools preparing and or implementing their 
CSR plans. (WestEd) 

Perhaps one of the greatest challenges of the CSRD program is its complexity. In 
addition to being asked to develop plans that include technology, 
parent/community involvement, professional development, and a research-based 
model of reform, schools are asked to intuit how these elements mesh together to 
form a comprehensive plan for reform. (WestEd) 

In five schools (of 12), the state and district provided extensive assistance to the 
school during the planning process. School districts analyzed student achievement 
data for these schools as well as other pertinent needs data. Districts also provided 




14 



19 



/ 



technical assistance in writing the grant, as well as functioning as a liaison 
between the school, model developers, and the community. In some cases, the 
state also provided assistance in the form of an external facilitator who was 
assigned to the school via another project. The external facilitator participated 
extensively in the research and analysis process. (AEL/CREP) 

Two laboratories noted the role of districts in providing administrative support such as 
negotiating with model service providers and budgeting know-how. WestEd researchers noted 
how schools appreciated this help from districts, while PREL research noted that “paperwork” 
between schools and model service providers was an implementation problem. 

From the initial analyses of our research, we learned the decision making process 
for schools was best aided by districts when 

• districts provided information to assist schools in their model adoption, 
negotiation of contract with developers, and facilitation of 
billing/materials associated with models; and 

• informed schools about reallocation of resources. (WestEd) 

Additionally, schools talked about the ways in which districts facilitated 
contractual negotiations between the school site and the model developer. 
(WestEd) 

Biggest administrative hurdles to implementation: 

• Contract writing, financial planning 

• Contract approval forms, including Sole Source approval, tax clearance, 

Board of Education approval. (PREL) 

The relationship between district support and model implementation is not clear. 

At the present, there is no evidence that the amount of district support has made 
any difference in implementation of CSR models. (AEL/CREP) 

School-Level Findings 

School Interface with Model Providers. An issue that was briefly touched upon by some 
researchers was barriers (both physical and logistical) to collaboration between schools and 
model providers. 

Some of the Maine schools that did not identify the Coalition of Essential Schools 
as their “model,” made this “choice” because the Coalition’s support 
infrastructure was too distant from their school. The Southern Maine Partnership 
(a Coalition of Essential Schools regional center), which is based at the University 
of Southern Maine, is nine hours from Caribou, one Maine CSRD recipient site, 
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and more than five hours from several others. Reflecting both the unavailability of 
Coalition support and the disposition for local autonomy, these “too-distant” 
schools all opted to implement locally designed models. As an aside, the 
experience of these “too distant” schools, all were rural, highlights a possible 
national issue regarding CSRD and rural schools; namely the dramatic limitation 
of model provider choices because of such geographic inaccessibility. A 1998 
LAB at Brown survey identified several model developers who were unable or 
unwilling to have their model implemented in Puerto Rico. (LAB at Brown) 

Biggest hurdles to implementation: 

• Long distance from technical service providers 

• Materials arrived late . . . (PREL) 

Model Selection at the School Level. A key issue discussed in the research of several 
participating laboratories is the level of teacher participation in model selection. Two 
laboratories specifically looked at the relationship between teacher participation and either 
school climate or model implementation. SERVE described three different decision mechanisms: 
Administrative; Team/Committee; and Whole School. AEL/CREP described two decision 
mechanisms: Top-Down and Participatory Decision-Making (PDM) for selection of models that 
were not locally developed. 

AEL/CREP and SERVE used the same survey instruments to examine school climate, 
and level of reform as outcomes of the model selection process. AEL/CREP found a relationship 
between the style of planning engaged in and the nature of the subsequent implementation. A key 
question that has not yet been completely addressed, because of the relatively early stage of the 
current CSRD implementation, is the link between decision making processes, model 
implementation and student performance. 

Six schools used a participatory decision-making approach to choose a model. In 
these schools, a committee or group of teachers was selected to research and 
assess models; using school needs analysis data and other instruments.... 
Teachers would typically choose a set of CSRD models for further research and 
analysis... The next round of model evaluation typically included teachers doing 
extensive research on a smaller number of CSRD models, which sometimes 
included visits to other schools using the model. Teachers would then present 
their findings to the entire faculty, who would then vote on the model that best 
met the needs of the school. (AEL/CREP) 

The second approach to choosing a model was much less participatory, although 
it was sometimes couched in participatory terms. Four schools in the study used a 
Top-Down approach. In these schools, only one CSRD model was presented to 
teachers for their consideration. Teachers were either not aware of additional 
models or had no opportunity to consider additional models (i.e., no opportunity 
was mentioned in the interviews or application document). (AEL/CREP) 
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The cross-laboratory pilot question asked if the planning and model selection 
process had an impact on CSR implementation. In order to address this question, 
teacher questionnaires were analyzed to determine if teacher responses at schools 
using a Top-Down approach differed from responses from teachers at schools 
using a PDM approach. (AJEL/CREP) 

Responses on the Comprehensive School Reform Teacher Questionnaire 
(CSRTQ) from Top-Down schools and Participatory Decision-Making schools 
were compared using analysis of variance. Specific teacher questionnaire items 
were chosen for the analysis due to their conceptual link with a participatory 
decision making approach. The response scale used ranged from strongly disagree 
(1) to strongly agree (5). Results from the analysis of variance are reported in 
terms of the F ratio, and the significance level (p) are reported in Table 4. 
(AEL/CREP) 

The same pattern of differences between Top-Down and Participatory Decision- 
Making schools was true for the Parent/Community Involvement Benchmark. On 
this benchmark, the low end of the scale indicated limited parental/community 
involvement, and the high end of the scale indicated parents being active 
supporters, and faculty and staff regularly plan opportunities for quality 
parent/community participation. These are reported in Table 3. (AEL/CREP) 

Table 3. 

Comparison of Participatory Decision-Making and Top-Down Approaches 

CSRTQ and Implementation Measures 

N=253, 1999-2000, (AEL7CREP) 



Questionnaire Item 


Mean 

Top-Down 

Approach 


Mean 

Participatory 

Approach 


F 


Sig. 


1. 1 have a thorough understanding of this school’s 
comprehensive school reform (CSR) program 


3.4 


3.7 


4.18 


.042 


12. Students in my class spend much of their time working in 
cooperative learning teams 


3.6 


3.9 


4.16 


.043 


17. Community support for our school has increased since 
comprehensive school reform has been implemented 


3.0 


3.4 


10.46 


.001 


19. Teachers are more involved in decision making at this 
school than they were before we implemented 
comprehensive school reform 


3.1 


3.4 


6.63 


.011 


21. Because of our school’s program, teachers in this school 
spend more time working together to develop curriculum and 
plan instruction 


3.3 


3.7 


8.87 


.003 
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Questionnaire Item 


Mean 

Top-Down 

Approach 


Mean 

Participatory 

Approach 


F 


Sig. 


22. Teachers in this school are generally supportive of our 
CSR program 


3.5 


4.0 


14.72 


.000 


27. My school receives effective assistance from external 
partners (e.g. university, businesses, agencies, etc.) 


2.9 


3.3 


6.31 


.013 


Implementation Measures 




Curriculum Benchmark 


2.5 


2.8 


5.68 


.018 


Parent/Community Involvement Benchmark 


2.2 


2.6 


4.36 


.038 



A similar analysis was conducted using the School Climate Inventory. Eight items 
on the Inventory were chosen for analysis due to their conceptual link with a 
participatory decision-making approach... Six items were related to parental and 
community involvement in the school. On three items, teacher responses from 
Participatory Decision-Making schools were significantly higher than those from 
Top-Down teachers. Differences on the remaining three items were non- 
significant... Two items on the Inventory addressed the issues of faculty and staff 
cooperation, and teacher involvement in decision making. These results are 
reported in Table 4. (AEL/CREP) 

Table 4. 

Comparison of Participatory Decision-Making and Top-Down Approaches To Model 

Selection on the SCI 

N=243, 1999-2000 (AEL/CREP) 



Questionnaire Item 


Mean 
Top Down 
Approach 


Mean 

Participatory 

Approach 


F 


Sig. 


5. Community businesses are active in this school 


2.9 


3.4 


9.2 


.003 


1 1. Parents are involved in a home and school support 
network 


3.2 


3.1 


.37 


.543 


18. Parents are invited to serve on school advisory 
committees 


3.9 


4.1 


5.5 


.019 


19. Parent volunteers are used whenever possible 


3.9 


4.0 


.50 


.483 


28. Faculty and staff cooperate a great deal in trying to 
achieve school goals 


3.8 


4.1 


6.7 


.010 


31. Teachers do not participate enough in decision 
making 


2.7 


2.8 


.03 


.856 
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Questionnaire Item 


Mean 
Top Down 
Approach 


Mean 

Participatory 

Approach 


F 


Sig. 


32. Information about school activities is 
communicated to parents on a consistent basis 


3.9 


4.1 


1.6 


.202 


37. Parents are often invited to visit classrooms 


3.6 


3.9 


3.9 


.048 



Although AEL/CREP found a relationship between the model selection process and both 
implementation and school climate, researchers at SERVE did not perform the same statistical 
tests and thus did not report similar findings from their data. These different findings may be 
partially due to the use of different constructs in describing the decision-making mechanisms. 

We hypothesized that schools using the more inclusive and democratic Whole 
School approach would tend to engender greater teacher buy-in to the 
comprehensive reform design, and increase awareness of the planning and 
implementation processes. The sub-groups are as follows: 

• Administration: Seven schools indicated that the administration or 
individuals outside of the school selected the comprehensive school reform 
model. One school was excluded from the comprehensive school reform 
teacher questionnaire analysis due to missing data. 

• Team/Committee: Fifteen schools indicated that a small group or committee 
selected the comprehensive school reform model for their school. 

• Whole School: Fourteen schools indicated that the entire school’s faculty and 
staff participated in the selection process for the comprehensive school reform 
model. However, one schools was excluded from the analysis of the 
comprehensive school reform teacher questionnaire items because no 
questionnaires were submitted. Additionally, two schools were excluded from 
the analysis for the school climate inventory because of missing data. 
(SERVE) 

For this preliminary examination of outcomes, we examined the descriptive 
outcomes for meaningful trends and patterns. The means shown in Table 5 were 
derived from a three-point scale, with 3 indicating agreement and 1 indicating 
disagreement. A response of 2 was considered neutral. For both of the questions 
from the teacher questionnaire, the Whole School process schools had a slightly 
higher mean. The Administration process was consistently rated the lowest. 
However, the effect sizes for the Whole School subgroup advantage were in the 
+.20 - +.25 range, indicating a relatively weak effect. (SERVE) 



Table 5. 

Mean Ratings of Decision-Making Involvement on Teacher Questionnaire 

N=1 025, 1 999-2000 (SERVE) 



Question 


Administration 


Team/ 

Committee 


Whole 

School 


Total 




Mean 


SD 


Mean 


SD 


Mean SD 


Mean 


SD 


Teachers are 
more involved in 
decision making 
at this school 
than they were 
before we 
implemented 
comprehensive 
school reform. 


2.12 


.85 


2.16 


.79 


2.32 .79 


2.21 


.80 


Teachers in this 
school are 
generally 
supportive of our 
CSR program. 


2.48 


.74 


2.54 


.65 


2.65 .62 


2.57 


.66 



In other studies, laboratory researchers observed different model selection processes, with 
some additional insights into top-down approaches and also subsequent attitudes and behaviors. 
As with the AEL/CREP and SERVE research, these studies also found different responses in 
schools to top-down decision-making. In some instances teachers appear to have (slowly) 
embraced principal-mandated reforms. Differences in school leadership and/or regional cultures 
also may explain some of the differences in findings across CSRD site. 

District culture varied across sites. In some districts, CSR model selection 
appeared to be site-based, in others, model selection appeared more top down, an 
integral part of district-wide initiatives. In the former districts, central office 
personnel played supportive, information providing, guiding roles. (McREL) 

Within school buildings, the majority of sites experienced resistance to change 
among staff; however, a couple of principals commented on how resistance or an 
attitude of compliance dissipated after implementation began. (McREL) 

The study was hot designed, nor was evidence collected, to determine under what 
conditions a top-down or bottom-up approach is more effective for improving 
schools and student achievement. (McREL) 

Leadership and management style strongly influenced how decisions were made. 

In both schools the leadership styles were very different but had an adverse effect 
on the staff’s active involvement in the entire planning process. The decision- 
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making process at both sites were exclusive in nature. Opportunities were few or 
not genuine in planning and decision making, resulting in the staff placing the 
locus of control outside of themselves — feeling “done to” rather than engaged in 
the process. (NWREL) 

The LAB at Brown reported a schism created by top down behavior at a school that was 
not successful in its bid for CSRD funding. This research project also described a slightly 
different meaning to the phase “top-down.” 

At an unsuccessful applicant school, the local control of the CSRD grant 
development process by a small cadre of ‘change-oriented’ faculty (with the 
principal’s vague blessing) had become grounds for a faculty schism. When the 
grant was not funded, this group (that wrote the grant proposal) was nonetheless 
initially still optimistic because of various promises that Promising Futures would 
be implemented anyway at the school. This group subsequently became frustrated 
when in their minds the principal reneged on that promise. In their minds it was a 
top-down decision to do nothing. This highlights how complaints about “top- 
down” methods can be directed at in-building processes as well as to mandates 
from district, state, or federal levels. It even suggests that external programs can 
be sought as a way of averting top-down structures. (LAB at Brown) 

Research by the laboratories is likely to continue on the relationships between decision 
making and outcomes. 

School Capacity to Reform. School preparedness for reform was addressed in NWREL’ s 
work with schools that did not receive CSRD grants and can also be derived from some of the 
LAB at Brown’s research that has already been presented. 

The most significant lesson in our preliminary data analysis is the necessity of 
preparing for change. This involves creating an internal catalyst for change where 
the locus of control is within the school community. Once the school staff 
recognizes the need for change, the next step is to develop a climate that fosters 
change. The two sites we are working with have a history of chaos and poor 
performance. The time it has taken to examine the professional working culture of 
each school and then to begin reshaping the organizational climate to support 
comprehensive reform has been lengthy. However, without addressing the 
readiness for change and investing into constructing a school culture and climate 
that advocates shared decision making and planning, change will be fleeting, 
replaced by the next a new idea, principal, or reform effort that is imposed on the 
school staff and community. The key issues that have percolated from our data 
analysis and influence a school’s readiness to undertake comprehensive school 
reform are the following: 

• Quality and stability of leadership 

• Ability of the staff to collaboratively work as a team 
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• Environment of respect and trust 

• A clearly defined and shared vision 

• Embracing high expectations 

• Desire and willingness to change among the staff (which influences degree of 
commitment) 

• External technical assistance and support 

• Accountability. (NWREL) 



CONCLUSIONS 

The various laboratories research projects about CSRD have taken many approaches and 
perspectives. This attempt to look for common themes across projects has revealed a wide 
variety of ways that states and districts support and mold CSRD. It makes clear that 
understanding CSRD planning requires consideration of roles and actions across multiple levels 
within the education system. 

Some states clearly integrate or align CSRD with other reform efforts. District roles in 
planning comprehensive school reform varied, including mandating models for schools, 
supporting school searches for appropriate models and support help in writing proposals, and 
assistance with contract negotiations. Many researchers examined the process used by school to 
select designs. A key issue of interest in several laboratory studies was the level of participation 
of school faculty in model selection. Each laboratory has categorized this level of participation in 
different ways. A key challenge for researchers looking at school-level decision making is to 
develop common definitions for the various levels of teacher participation in model selection and 
other key management decisions. 

A key issue that is not completely understood is how alignment of CSRD within existing 
policy frameworks and institutional roles interacts with the model options available to schools 
and the success of CSRD implementation. State-level alignment and its effects are evident. 
Maine’s alignment of CSRD with existing reform efforts limited CSRD to secondary schools and 
narrowed the potential model choices for schools, yet added coherence to the rollout. This 
allowed participating schools to easily learn from each other. California’s requirement of district 
and school applications gave a de facto advantage to districts that had existing reform plans and 
structures for facilitating schools’ efforts. 

The importance of alignment of policy frameworks and institutional roles between 
districts and schools is not as clear. It is clear that districts served many different roles, and 
schools used many different model selection processes. However, the evidence is mixed on the 
how district roles and school-level model selection processes affected implementation. 

Work with schools that were not successful in their CSRD applications showed that 
schools must develop a capacity for reform. Next questions for investigation may be to examine 
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which district roles are appropriate, given different school-level capacities to make reform 
decisions. In other words, given the wide variety of district support and decision-making 
processes, is it likely that some types of district support are more appropriate for different levels 
of school decision making and reform capacity? District policy frameworks for schools with 
substantial reform capacity may include the ability to temporarily allocate resources for grant 
competitions and administrative support; district policy frameworks for schools with less 
capacity may be more directive and/or developmental. 
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APPENDIX 
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moatss@ael.org 



University of Memphis 
302 Browning Hall 
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During the 1999-2000 academic year, AEL, Inc. and the Center for Research in 
Educational Policy (CREP) at the University of Memphis collaborated to conduct research in 12 
Comprehensive School Reform Demonstration (CSRD) schools located in Kentucky, Tennessee, 
Virginia, and West Virginia. These schools were chosen to represent a wide variety of urban and 
rural locations as well as a variety of CSRD models. Data gathered at the schools included 
principal interviews, classroom observations, teacher focus groups, faculty questionnaires, 
CSRD Applications, and implementation benchmarks. 

Findings from the first year of the research study include (1) the amount of support 
provided to schools by the district varied greatly, (2) all schools reported needs analysis 
information in the CSRD application document, however, it was difficult to determine if the 
needs analysis information was actually used during the planning and model selection process or 
if it was simply added to the CSRD application document post facto, (3) teachers at schools 
using a participatory decision-making approach to model selection reported significantly higher 
levels of support for the model than did teachers at schools using a tops down approach, and (4) 
schools using a participatory decision-making approach reported significantly greater progress in 
implementing the model than did schools using a top down approach. 

A related technical report will be available on the AEL website (www.ael.org) in early 
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Dr. Ted Hamann, a Research and Development Specialist at the LAB, authored all the 
initial reports and subsequent editing that constitute the LAB’s contribution to this cross-REL 
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study. However, LAB colleagues Brett Lane, Matt Hudak, Ivana Zuliani, and Patti Smith have 
all carried out substantive CSRD work upon which he draws. 

Dr. Hamann’s roles in relation to CSRD have included both research and technical 
assistance, with research being his predominant role in Maine and technical assistance provider 
being his main role in Puerto Rico and in New Hampshire. However, his Maine role did include 
some technical assistance and a summary of his visits to CSRD schools in Puerto Rico is the 
basis for a technical report he has prepared for OERI. 

With planning processes as the obvious focus, the segments of LAB research presented 
here look in particular at the interface between federal initiators of CSRD and state departments 
of education and at the interface between state departments of education and schools. However, 
the LAB’s studies in Puerto Rico have also considered the interface between schools and model 
providers. For reasons discussed in the text, that is not a pertinent concern for Maine because of 
the particularities of its CSRD roll-out strategy. 

The LAB has been conducting an ethnography-oriented statewide case study of CSRD 
implementation in Maine since February 2000. This research, also describable as an 
“ethnography of education policy”, uses a framework outlined by Levinson and Sutton (in press). 
The LAB’s research in Maine has included site visits at 14 schools (8 of them CSRD schools), 
evaluation of six CSRD schools’ school portfolios, review of site documents (ranging from 
schools’ CSRD applications to the state CSRD coordinator’s formative evaluation frameworks), 
facilitation at a workshop for all 11 Maine CSRD schools, attendance at a summer academy that 
highlighted changes underway at several CSRD schools, joint presentations at three national 
meetings with Maine educators involved in CSRD, and collaboration with developers of Maine’s 
Promising Futures secondary reform framework who helped push for that state’s linkage of 
CSRD and Promising Futures. The LAB CSRD work in Puerto Rico involved a March 2000 
training workshop for school site visitors/evaluators and two September 2000 trainings on 
school-site self-evaluation. During their trips to Puerto Rico for the workshops, LAB staff were 
able to visit six CSRD implementation sites in what are best described as ‘ visitas de conocer’ 
(i.e., visits to gain familiarity). The New Hampshire reference in the report comes from 
information gathered during a school self-evaluation workshop with all six New Hampshire 
CSRD schools. 

From the above activities, the LAB has drawn numerous conclusions, many 
provisionally. However, based on their work, Dr. Hamann and his colleagues are confident of 
their claim that state-level interpretations, adaptations, and resulting supports of CSRD are 
critical for CSRD planning, alteration, and enhancement. Adjustments at the state level make 
CSRD implementation at the local level more responsive to state and local contextual factors and 
thereby more viable. State-level adjustments also mean that CSRD looks substantively different 
in different jurisdictions. 

Reference 

Levinson, B. A. & Sutton, M. (in press). Policy as/in practice: Developing a sociocultural 

approach to the study of educational policy. In Sutton, M & Levinson, B.A. (Eds.) Policy 
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as practice: Toward a comparative sociocultural analysis of educational policy. 
Westport, CT: Ablex Press. 

LAB technical reports on CSRD in Maine and Puerto Rico will be available after December 10, 
2000. For requests contact Ted Hamann (Edmund Hamann@brown.edu) or Brett Lane 
(Brett Lane@brown.edu) . 
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In winter 2000, McREL conducted a multiple case study of nine CSRD-funded schools in 
four central region states. The study focused on classroom practices in K-5 literacy in association 
with CSR model orientation, namely, whether the model focused on literacy or was more 
generally focused on school culture and developing human potential. In addition to surveying 
teachers about their use of classroom literacy practices, school and district administrators were 
interviewed about the role of district support for school reform. This report describes findings 
from these administrator interviews. The role of districts as information providers and guides and 
supporters of the grant application process was evident across the majority of sites. Specifically, 
both locally-designed and state-directed needs assessment and school improvement processes 
prepared local school teams to successfully apply for CSRD program participation. 
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This document presents the results of program evaluation research investigating 
Comprehensive School Reform Demonstration Project (CSRDP) planning and its effect on the 
quality of reform implementation. 

; 

Three phases of the formative evaluation of CSRDP are addressed: technical assistance, 
demonstration site selection, and early implementation. Technical assistance was provided by the 
Hawaii State Department of Education (HIDOE), model program developers, and PREL. To 
assess satisfaction with services received in preparation for and throughout the process of early 
program implementation, administrators of schools receiving grants were interviewed to assess 
their satisfaction. Provisions for technical assistance from each of the service providers were 
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found to be good, with recipient ratings evidencing high levels of satisfaction with pre-proposal 
assistance and post-funding provisions. Among reported difficulties in carrying out 
programming, paperwork issues in the form of completing contracts and approvals were 
repeatedly described. These responses evidence a potential benefit to future recipients of CSRDP 
funds of increased technical assistance training and support in this area. 

The second phase of the CSRDP formative evaluation involves the degree to which the 
protocol for the selection of demonstration program sites was followed. An inspection of ratings 
of the model programs were found to be carried out appropriate to the intentions stated in the 
state’s federal CSRDP grant proposal. 

I 

The third phase in the formative evaluation of CSRDP is the early implementation of 
model programs conducted since the period of grant awards (January 1999). To measure the 
fidelity of early CSRDP implementation, HIDOE site visitation measures and decisions 
regarding the overall progress of schools were observed. An analysis of the year one 
implementation of CSRDP schools revealed that the major events carried out by CSRDP schools 
included the receipt of model developer trainings, HIDOE/PREL technical assistance, attendance 
at CSRDP-related conferences, and planning activities. In the first year, eight of the ten CSRDP 
schools were found to be in the process of planning their CSRD program, while the remaining 
two initiated implementation. 

Preliminary year two evaluation measures indicate that CSRDP implementation is 
proceeding as planned for more than half of the sites (56%). Delays in implementation for all 
sites have occurred as a result of insufficient time for planning, articulation, and teacher 
preparation; unavailability of program materials; inadequate professional development; program 
ambiguities; lack of teacher commitment; and the need to modify elements of the program to 
meet school needs, among others. Preliminary findings also indicate that the integrity of 
implementation is highest for schools with reform models having the highest levels of 
programmatic structure (e.g., Success For All, America’s Choice), though high structure is 
sometimes associated with implementation delays. 
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The purpose of this in-depth study is to document, analyze, and report about how high 
poverty, low-performing schools successfully restructure themselves and utilize limited 
resources in order to support and sustain comprehensive reform efforts that focus on improving 
reading achievement for all its students. NWREL selected a rural k-8 school on the Oregon 
coast and an elementary school in an agricultural community in Washington as partner sites for 
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this case study. NWREL has been drawing on its internal expertise and services to provide 
information, technical assistance, staff development and other resources to assist these schools in 
their efforts to fully implement a research-based reading system that is embedded in 
comprehensive school reform. 

This case study addresses the blind spot in the majority of comprehensive school reform 
research by investigating what is needed to create a fertile school environment that initiates and 
sustains schoolwide reform efforts. The case study schools provided critical knowledge of what 
it takes to effectively restructure a high poverty, low performing school. The research reflects 
one constant theme: schools need to prepare for change by first creating a positive, supportive 
environment of respect and trust. This theme and its key features are found on every 
researcher’s list of essential ingredients for successful reform. What has been absent from the 
implementation of reform is the time and devotion necessary to truly concentrate on this crucial 
first step. People, not programs, make change happen. 

A report titled “Programs Don’t - People Do Insights into Schoolwide Change” will be 
available December 1, 2000. In this report informs and assists beleaguered schools to 
successfully negotiate the difficult and complex process of change, most specifically, as they 
develop a comprehensive schoolwide literacy program. 
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Report Summary prepared for SERVE by Steven M. Ross, Marty Alberg, Robert Horn, 
Deborah Lowther, and Lana Smith at the Center for Research in Educational Policy (CREP), The 
University of Memphis 

The SERVE study of Comprehensive School Reform Demonstration (CSRD) schools 
consisted of examining processes and preliminary outcomes regarding design selection, teacher 
and administrator support, school climate, implementation process, and teaching strategies at 39 
of the 333 CSRD school in Alabama, Florida, Georgia, Mississippi, North Carolina and South 
Carolina, the six states of the SERVE region. 

Selected questions from the Comprehensive School Reform Teacher Questionnaire 
(CSRTQ) and the School Climate Inventory (SCI) were considered relevant to the present 
analysis interests. A review of the CSRTQ and SCI was conducted to determine the questions 
that were logically relevant to assess comprehensive school reform implementation, program 
support, and inclusion in decision making. 




29 



34 



Qualitative comments provided by Site Researchers and process records for each of the 
CSRD schools were used to determine inclusion of schools into one the three design selection 
sub-groups: Administration, Team/Committee, and Whole School. 

Based on the analysis, the following overall findings emerged: 

• There appears to be general support by the teachers for their CSRD program 

• Teacher involvement in decision making and their level of trust in the schools was 
relatively low 

• Commitment, achievement, and review of school goals appear to be strong areas in most 
schools 

As the longitudinal study continues through the 2002 school year, SERVE will obtain 
further data regarding the implementation and success of the varied comprehensive school 
reform designs. A related regional report will be available in December 2000. 
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This study looks at the ways in which nine districts in California supported schools from 
the pre-application phase through the first year of implementation of comprehensive school 
reform. Our major finding is that district staff worked in a variety of different ways with schools. 
We identified two different typologies to best characterize different approaches to supporting 
schools. Our rationale for creating typologies is to provide a common language to educators 
working with reform. 

From this study we learned that over time there was a developmental process between 
both the schools and the districts. In the pre-application phase the majority of schools that 
developed the highest scoring plans worked closely with district staff and/or staff hired in from 
the outside paid for by the district. In the implementation phase the majority of schools that 
reported the greatest sense of satisfaction with their plans were schools that maintained their 
initial relationship with the district but the district itself was less proactive and more reactive to 
specific school requests. 

Through our work we learned districts can support schools both through intervening, 
proactive measures as well as waiting and reacting to individual school needs. Through both 
proactive and reactive work districts increased focus on student achievement by reducing the 
number of distractions that would otherwise divert energy of principals and teachers. Districts 
can remove distractions by reducing the extent to which central office staff plays rule-monitoring 
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roles. With fewer central office dictates, principals and teachers have more time to address 
instructional improvement. Similarly, district staff may be required to work in ways that reduce 
many of the bureaucratic or auxiliary functions that tended to divert the attention of school 
personnel away from teaching and learning. As we learned districts helped schools build their 
capacity to change by utilizing a combination of proactive and reactive techniques to best 
facilitate comprehensive school reform. 

Concurrent with the research we also learned that a dual approach (providing proactive 
support as well as reactive) to supporting schools worked well for a number of sites. In some 
instances districts offered a great amount of support up front through the application phase and 
then let the schools take over to implement their plans. For some schools this worked very well, 
while in other schools, staff reported they would like more support from the district. Although 
there were some complaints about the reactive approach it seemed to be well received by the 
majority of our sites during the implementation phase, and also the most realistic option given 
the human and financial resources available at the district level. 

A less successful strategy and certainly one that caused confusion at the school sites was 
a reactive approach to working with schools both in their applications for and implementation of 
CSRD. There are several reasons why districts adopted this behavior. In two cases districts 
started out as very proactive, but when key staff members left there was no one in place to take 
over their scope of work therefore necessarily forcing the schools to take over and manage for 
themselves. In other cases districts opted to rely on the support of model developers in 
implementing CSR plans. 
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